Papers with text modeling
Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling (N19-1)
Copied to clipboard
| Challenge: | Empirical experiments show that our model learns latent distributions that respect latent space geometry and is able to generate sentences that are more diverse. |
| Approach: | They propose a Variational Wasserstein Autoencoder with Riemannian Normalizing Flow to solve this problem by transforming a latent variable into a space that respects the geometric characteristics of input space. |
| Outcome: | Empirical results show that the proposed model avoids KLvanishing and has better performance in language modeling, likelihood approximation, and text generation tasks. |
Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to model long documents are difficult due to the quadratic complexity of text length. |
| Approach: | They propose a hierarchical interactive Transformer for efficient long document modeling. |
| Outcome: | Extensive experiments on three benchmark datasets validate the efficiency and effectiveness of Hi-Transformer in long document modeling. |
Cross-Media Keyphrase Prediction: A Unified Framework with Multi-Modality Multi-Head Attention and Image Wordings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on text modeling, ignoring the rich features embedded in the matching images. |
| Approach: | They propose a novel multi-modal multi-head attention model to capture cross-media interactions and image wordings to bridge the two modalities. |
| Outcome: | The proposed model outperforms the current state of the art based on text modeling and image matching . |
On the Encoder-Decoder Incompatibility in Variational Text Modeling and Beyond (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work has shown that the optimization of variational autoencoders suffers from the posterior collapse problem. |
| Approach: | They propose a variational autoencoder that couples a VAE model with a deterministic autoencoding model and improves the parameters via weight sharing and decoder signal matching. |
| Outcome: | The proposed model improves on benchmark datasets and improves diversity of dialogue generation. |
Investigating Capsule Networks with Dynamic Routing for Text Classification (D18-1)
Copied to clipboard
| Challenge: | Earlier efforts in text modeling have achieved limited success on word meanings . convolutional neural networks (CNNs) are used to model higher level concepts and facts in texts . |
| Approach: | They propose three strategies to stabilize dynamic routing process to alleviate disturbance of noise capsules. |
| Outcome: | The proposed methods achieve state-of-the-art on 4 out of 6 datasets . they show that capsule networks exhibit significant improvement over baseline methods . |
Time-Aware Language Modeling for Historical Text Dating (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to automatic text dating ignore diachronic change of words, which may affect the efforts of text modeling. |
| Approach: | They propose a time-aware language model to learn temporal word representations by transferring language models of general domains to those of time-specific ones and build a hierarchical modeling approach to represent diachronic documents by encoding them with temporal representations. |
| Outcome: | The proposed model outperforms state-of-the-art approaches in historical text dating and other NLP tasks. |
ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing omni-multimodal large language models lack incomplete modality support or lack autonomous proactive monitoring. |
| Approach: | They propose a real-time omni-multimodal assistant for unified reactive and proactive interaction that decouples response initiation from generation to ensure precise triggering without task conflict. |
| Outcome: | The proposed model achieves state-of-the-art performance on proactive tasks while competing in reactive settings. |